UNITED STATES PATENT APPLICATION FOR: 
FACE DETECTION METHOD AND APPARATUS 

INVENTOR: 
CHENGJUN LIU 



KAPLAN & GILMAN, LLP 
Attorney Docket No. 436-5 



Attorney Docket No. 436/5 



FACE DETECTION METHOD AND APPARATUS 

Related Application 

[0001] This application claims the benefit of priority to provisional 
apphcation serial number 60/446,596 ("the *596 appHcation"), filed February 11^**, 
2003. The '596 application is hereby incorporated by reference in its entirety. 

Technical Field 

[0002] This invention relates to image analysis, and more particularly, to an 
improved method and apparatus for detecting the presence of faces within an image 
while minimizing the required calculations and maximizing speed and efiBciency. 

Background of the Invention 

[0003] Face detection has a variety of applications in government, security, 
etc. Face detection involves the ability of software and apparatus to examine a 
composite image and detect the presence of faces within the image. Face detection 
algorithms are known in the art and are computationally expensive. A variety of 
techniques utilized in prior art face detection algorithms are described and cited as 
references 1-28 in the '596 provisional application. One of the main challenges in 
face detection technology is the ability to discriminate between face and non face 
images given that the faces may be of any size and located anywhere in the image. 

[0004] Earlier efforts have been focused on correlation or template matching, 
matched filtering, subset methods, deformable templates, and similar such techniques. 
One representative technique, in order to cope with the variability of face images, 
utilizes six Gaussian clusters to model the probability distributions for face and non- 
face patterns, respectively. The density fimctions of the Gaussian distributions are 
then fed through a multilayer perceptron classifier for face detection. Such 
techniques are extremely computationally expensive, and subject to relatively high 
rates of false detection. 

[0005] It is an object of the invention to decrease the computational cost of 
performing face detection; 

[0006] It is also an object of the invention to provide a face detection 
algorithm that has an acceptable false detection; 
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[0007] It is also an object of the invention to diminish the large number of 
probability density functions utilized in face detection algorithms; 

[0008] It is still a further object of the present invention to provide a relatively 
accurate face detection algorithm. 

Summary of the Invention 

[0009] The above and other problems of the prior art are overcome in 
accordance with the present invention, which relates to a novel representation of the 
input images and the processing of such novel representation using a novel Bayesian 
classification technique in order to detect faces. 

[0010] In accordance with the invention, a discriminating feature analysis 
(DFA) procedure is carried out and DFA vectors are calculated on a variety of 
training images. The DFA vectors are then utilized to model the face and non-face 
classes using, for each class, a single multivariate Gaussian probability density 
function (PDF). The generation of the DFA vectors and the modeling of the face and 
non-face classes based upon such DFA vectors renders training complete. 

[0011] Thereafter, an input image is processed to calculate the DFA vectors 
associated with such input image. The DFA vectors are then processed through a 
special Bayesian classifier as discussed more fully below in order to control the false 
detection rate and provide optimal accuracy in performing face detection. The output 
of the foregoing then classifies portions of the image as face or nonface with a 
relatively high degree of accuracy and low probability of false detection. 

[0012] The DFA representation may itself be used in representations other 
than for face detection algorithm. 

Brief Description of the Drawings 

[0013] FIG. 1 is a flow chart of the basic steps of the face detection algorithm 
of the present invention; and 

[0014] FIG. 2 shows the discriminating feature analysis (DFA) of face and 

nonface classes, (a) The first image is the mean face, the second and the third images 

are its 1-D Harr wavelet representation, and the last two bar graphs are its amplitude 

projections, (b) The mean nonface, its 1-D Harr wavelet representation, and its 

amplitude projections. Note that the images and projections in (b) resemble their 

counterparts in (a) due to the fact that the nonface samples lie close to the face class. 
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Detailed Description of the Preferred Embodiment 

[0015] FIG. 1 depicts a flow chart of the basic technique implemented by the 
present invention. The calculations performed by the various portions of FIG. 1 are 
included in the provisional application that is incorporated herein by reference, and 
such mathematical equations will not be repeated again herein. 

[0016] In FIG, 1, the program is entered at start block 101 and training 
images are input into the system at block 102. In experiments performed in 
connection with the present invention, the training data for the Bayesian 
Discriminating Features methodology consisted of 600 FERET frontal face images 
that were used to model the face and nonface classes. The face class thus contains 
1200 face samples afl:er including the mirror images of the FERET data, and the 
nonface class consists of 4,500 nonface samples, which are generated by choosing the 
subimages that lie closest to the face class from 9 natural scene images. 

[0017] This training data is utilized at block 103 in order to compute the 
discriminating feature analysis (DFA) vector of the training images. The DFA vector 
is a novel featured vector with enhanced discriminating power for face detection. The 
DFA representation, shown for example in FIG. 2 hereof, combines the input image, 
its 1-D Harr wavelet representation, and its amplitude projections. The DFA 
representation of the training images may be calculated from Equation 6 in the 
provisional application incorporated herein by reference. The derivation of such 
equation is shown at pages 5-6 of said provisional. The output of Equation 6 
represents the combination of the image, its 1-D Harr wavelet representation, and its 
amplitude projections. 

[0018] Once the DFA vectors are calculated for the training images, the 
system stores the DFA vectors at block 104. Accordingly, by block 104, the entire set 
of training images has been processed and translated into DFA vectors so that the 
algorithm can next model the face class and non-face classes from these DFA 
representations. 

[0019] The modeling of the face and non-face classes is represented generally 
by operational block 105 in FIG. 1. The conditional probability density fimction of 
the face class can be estimated using a single multivariate Guassian distribution, 
rather than up to six Guassian clusters as utilized by most prior art systems. The 
monitoring of the face class is accomplished in accordance with Equation 13 in the 
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incorporated provisional application. That equation can be used to model the face 
class. 

[0020] Continuing with operational block 105, the nonface class modeling 
starts with the generation of nonface samples by applying Equation 13 to natural 
images that do not contain any human faces at all. Those subimages of the natural 
images that lie closest to the face class are chosen as training samples for the 
estimation of the conditional probability density function of the nonface class, which 
is also modeled as a single multivariate Gaussian distribution. The conditional 
density function of the nonface class is estimated as Equation 18 in the incorporated 
provisional application. Accordingly, at the completion of block 105, Equations 13 
and 1 8 have already been utilized to calculate the conditional PDFs for both face and 
nonface classes from a single multivariate Gaussian distribution. 

[0021] The PDFs are stored after being modeled at block 105. The stored 
PDFs render the system trained and ready to operationally detect faces in input 
images. 

[0022] Block 106 is the first step shown in FIG, 1 wherein an actual input 
image in which a face is to be detected is provided to the system. As previously 
noted, the system already has the PDFs of the face and nonface classes stored. Upon 
input of the image, the system calculates the DFA vector associated with the input 
image. That vector DFA must then be used to classify the image and portions thereof 
as face or nonface classes. 

[0023] To perform such classification, control is transferred to block 108 
where the BDF method applies the Bayesian classifier for multiple frontal face 
detection. The Bayesian classifier provided at Equations 19-25 of the incorporated 
provisional application is executed upon the DFA of the input image. The Bayesian 
classifier will then determine, according to Equation 25 of the incorporated 
provisional, whether the image or subimage being examined is a face or a nonface 
class. The equations 19-25 define what is termed herein a Bayesian classifier. 

[0024] Note there are two empirically derived control parameters in the BDF 
equations, x and 6, that must be finely tuned to the particular system under 
consideration. For the experiments conducted with respect to the present invention 
and described in the incorporated provisional application, the selected values of 300 
and 500, respectively, were found to give acceptable performance. However, trial 
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and error based upon the particular training images and anticipated input images may 
require these values to be adjusted slightly up or down. 

[0025] It is noted that the DFA vector shown in FIG. 2 combines the input 
image, its 1-D Harr wavelet representation, and its amplitude projections. The DFA 
vector may be useful in other image processing and detection systems, and is not 
limited to the face detection algorithms of the present invention. It is also noted that 
the empirically derived parameters discussed above may vary from system to system, 
and are not Hmited to those set forth herein or in the incorporated provisional. 
Various modifications or additions will be apparent to those of skill in the art, and 
such variations are intended to be covered by the appended claims. 
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